WORLD INTELLECTUAL PROPERTY ORGANIZATION 
International Bureau 




PCT 

INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) 



(51) International Patent Classification 7 ; 
G06K 9/20 



Al 



(11) International Publication Number: 



WO 00/05679 



(43) International Publication Date: February 200^(03,02.00) 



(21) Internationa) Application Number: PCT/US99/ 1 3734 

(22) International Filing Date: 17 June 1999 (17.06.99) 



(30) Priority Data: 
09/119,570 



20 July 1998 (20.07.98) 



US 



(71) Applicant: INTEL CORPORATION [US/US]; 2200 Mission 
College Boulevard, Santa Clara, CA 95052 (US). 



(rn~Tri^TntTTr* ^irHT™ , Cad}- 16) North Forest, Chandler, AZ 
— - 85226 (US). 

(74) Agent: TROP, Timothy; Trop, Pruner, Hu & Miles, P.C., Suite 
100, 8554 Katy Freeway, Houston, TX 77024 (US). 



(81) Designated States: AE, AL, AM, AT, AU, AZ, BA, BB, BG, 
BR, BY, CA, CH, CN, CU, CZ, DE, DK, EE, ES, FI, GB, 
GD, GE, GH, GM, HR, HU, ID, IL, IN, IS, JP, KE, KG. 
KP, KR, KZ, LC, LK, LR, LS, LT, LU, LV, MD, MG, MK, 
MN, MW, MX, NO, NZ, PL. PT, RO, RU, SD, SE, SG, SI, 
SK, SL, TJ, TM, TR, TT, UA, UG, UZ, VN, YU, ZA, ZW, 
ARIPO patent (GH, GM, KE, LS, MW, SD, SL, SZ, UG, 
ZW), Eurasian patent (AM, AZ, BY, KG, KZ, MD, RU, TJ, 
TM), European patent (AT, BE, CH, CY, DE, DK, ES, FI, 
FR, GB, GR, IE. IT, LU, MC, NL, PT, SE). OAPI patent 
(BF, BJ, CF, CG, CI, CM, GA, GN, GW, ML, MR, NE, 
SN, TD, TG). 



Published 

With international search report. 



(54) Title: DISPLAY^ CAPTl JRH SYSTEM 
(57) Abstract 

A computer system captures information from a tele- 
vision display digital signal. The digital signal is subjected 
to optical character recognition to allow the computer sys- 
tem to recognize characters displayed on that screen. The 
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Display Capture System 

5 Background 

This invention relates generally to computers, and in particular embodiments to 
capturing information from television or computer displays. 

Personal computers are now moving into the arena previously occupied by television 
receivers. The so-called set top personal computers interact with television receivers to 

10 provide advanced programming functions, digital television and other functions normally 

associated with personal computers, such as internet access and games. Thus, users of the set 
top personal computers may have at their disposal both a television capability and the 
capability normally associated with the personal computer. The set top personal computers 
use the television screen as the display both for the computer system and the television 

15 receiver. 

Computers may also display television pictures on a computer display or monitor. A 
tuner card in the computer acts as a television receiver so that a television program may be 
viewed on a computer monitor. 

Contact information is commonly displayed in television programming, particularly in 
20 television advertising. Many viewers have difficulty in recording contact information 
because, by the time they locate pencil and paper, the information is no longer displayed. 
Thus, advertisers may lose important opportunities and viewers may miss desirable 
information. 

Similarly, some contact information may be displayed as a graphical image (ie. in 
25 graphics rather than text mode) on an Internet web page. This may be the case where 
elaborate fonts or graphical displays are desired, for example with trademarks. 

It would be desirable to enhance the ability of a user of a computer system to retain 
non-textual information from a display. 

30 Summary 

In accordance with one aspect, a method for obtaining information from an image 
signal using a computer includes obtaining a digital representation of a display image. 
Characters in the image are recognized and the desired content within the image is located. 
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Brief Description Of The Drawing 

Figure 1 illustrates the display screen of a television showing an advertisement; 

Figure 2 is a file captured from information on the screen shown in Figure 1; 

Figure 3 is a file containing information captured from the screen shown in Figure 1 ; 

Figure 4 is a block diagram showing a computer system for implementing one 
embodiment of the present invention; and 

Figure 5 is a flow chart showing a flow for one embodiment of the present invention. 

Detailed Description 

Images containing contact information may be captured from a television picture or a 
computer display and manipulated by a computer system to facilitate contact with the 
broadcast content provider, which may be the advertiser on a television program. For 
example, television advertisers commonly broadcast advertisements like that shown in Figure 
1 which display a variety of information about their products. These advertisements may also 
include contact information such as names, addresses, telephone numbers and universal 
resource locators (URLs) which may be utilized to contact the advertiser or other content 
provider. 

A computer monitor display may also include images that include contact information 
in an image or graphics mode, for example using bit mapped graphics, rather than a text 
mode. The contact information may be included as part of an Internet web page, for example. 

In accordance with one aspect, the signal information that produces the image on the 
television or computer display 10 is digitally captured. For example, the telephone number 
depicted on the display 1 0 shown in Figure 1 may be recorded and provided in the form of a 
digital file 12 as indicated in Figure 2. Similarly, additional information from the screen 10 
may be captured and placed in an appropriate field 15 according to the type of contact 
information. 

S oftware may use logical heuristics to determine which information constitutes a 
person's name, which information constitutes a company name, which information 
constitutes an address, which information is a phone number and which information 
constitutes an Internet URL. Common characteristics of each of those fields may be stored in 
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a program to determine whether or not a given piece of information belongs in a particular 
field. 

Referring to ^tguc^i) a set top computer system 1 5 includes a processor 16 connected 
to system memory 18 and a north bridge 20. The north bridge 20 may conventionally 
5 connect to a bus 22 which may in turn be coupled to a display controller 24 which is in turn 
connected to a television 25. The bus 22 may also connect to a south bridge 26. The bridge 
26 may couple a hard disk drive 28 which may contain one or more programs 30 and 31 
which, in the illustrated embodiment could be an optical character recognition (OCR) and a 
telephone/address directory program. 

10 The south bridge 26 in turn connects to a bus 32 which may connect to a serial input 

output (SIO) port 34. The port 34 may connect to a keyboard 36, a teleph one dialer 40 an d a 
modem 38. The basic input output system (BIOS) 42 may also be connected to the bus 32. 
Of course, other implementations are possible and various additional components and 
peripherals may be connected to a conventional set top computer system. 

15 Referring now toT^^^s] the computer system 15 initially re ceives a request for 

capture services as indicated in block 44. This may be the result of the u ser clicking o n a 
button on a display screen 24 or by activating a particular h ot key on the k eyboard 36. This 
request is interpreted as a request to capture information from the television display. The 
image displayed at the time of the request may be digitally captured as indicated in block 46 

20 through the operation of the processor 16. The captured digital image is then subjected to an 
optical character recognition conversion, as indicated in block 48, using the optical character 
recognition software 30. This^softwar e scans the i nformation in the di gital imag e and 
identifie sand associates the image sh apes with known characters such as numbers and letters. 
It then provides a text mode representation of the image or graphics mode data. 

25 The computer system uses heuristics to locate information which corresponds to 

contact information styles, as indicated in block 50. For example, numbers followed by 
letters may be recognized as addresses while pairs of words with capital letters may be 
recognized as names. Common first names may be stored in a database to help determine 
that a personal name is involved. Similarly, particular sequences of numbers may be 

30 recognized as t elephone ji unibgrs and certain letter sequenc gsjnay be recognized as URLs. 

Oncgjhe ^ontact information has been identifie d, it may be displayed to the user, for I 
example, on the display screen, and the user may check the displayed information to insure I 
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that the desired information was captured, as indicated in block 52. The user can insert 
corrections in a corrections window (block 52) and the contact information is revised (block 
53). For example, if two telephone numbers are returned, the user can indicate which 
telephone number is desired. 
5 The selected contact information may then be stored in memory 18 or on the hard disk 

drive 28, as indicated in block 54. In some cases, it may be desirable to then make contact, as 
indicated in block 56, using the contact information. Th e dialer 40 may. be automatically 
activated to^glUhe .stored te lephone numher^Alternati vely , where a URL has been 
identified, the modem 38 may be used to automatically contact an appropriate Internet service 

10 provider and to obtain the desired web page. 

Contact information that has been captured off of the television display may then be 
automatically added to the database of conventional software 31 that compiles contact 
information, such as software telephone or address directories. Once the information has 
been identified and placed in the appropriate fields it can be supplied to such software 

15 directories for later use, as indicated in block 58. These directories may provide searching and 
autodialing capabilities and the like. 

The computer system 1 5 determines whether the telephone number is either a local 
number or a long distance number by comparing any area code with a predetermined local 
area code. If the call is a long distance call, a dialing prefix may be added. 

20 The present system facilitates making contact with information providers who provide 

content such as advertising over television broadcasts or graphics mode information on web 
pages. It also facilitates storing such contact information in contact software such as address 
or telephone directory software. 

While the present invention has been described with respect to a single embodiment, 

25 those skilled in the art will appreciate numerous modifications and variations therefrom. It is 
intended that the appended claims cover all such modifications and variations as fall within 
the true spirit and scope of the present invention. 
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What is claimed is: 

1 1 . A method for obtaining contact information from an image signal using a 

2 computer, comprising: 

3 obtaining a digital representation of a display image; 

4 recognizing characters in said image; and 

5 locating contact information in said characters. 

1 2. The method of claim 1 including locating a telephone number in said 

2 characters and automatically dialing said telephone number 

1 3. The method of claim 1 including locating an Internet address in said characters 

2 and automatically contacting said Internet address. 

1 4. The method of claim 1 including storing said contact information on the 

2 computer. 

1 5. The method of claim 1 including providing the contact information to the user, 

2 allowing the user to alter the contact information and revising the contact information 

3 according to information from the user. 

1 6. The method of claim 1 including recognizing a user command, and upon 

2 receipt of the user command obtaining a digital representation of the display image being 

3 displayed at the time the user command is received. 

1 7. The method of claim 1 wherein said contact information includes a telephone 

2 number. 

1 8. The method of claim 1 wherein said contact information includes an Internet 

2 address. 
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1 9. The method of claim 1 including automatically adding said contact 

2 information to a software address directory. 

1 10. The method of claim 1 wherein the obtaining step includes obtaining a digital 

2 representation of a television display image. 

1 11. The method of claim 1 wherein the obtaining step includes obtaining a digital 

2 representation of a computer display image in graphics mode. 

1 12. A computer system comprising: 

2 a processor; 

3 memory coupled to said processor; 

4 an input device coupled to said processor; 

5 a television receiver coupled to said processor to display television programs; and 

6 an optical character recognition device arranged to scan signal information for generating a 

7 television picture and to identify contact information in said signal information. 

1 13. The system of claim 12 including a telephone dialer coupled to said processor. 

1 14. The system of claim 12 including a modem coupled to said processor. 

1 1 5 . An article comprising a computer readable medium storing instructions to 

2 enable a computer to: 

3 obtain a digital representation of a display image; 

4 recognize characters in said image; and 

5 locate contact information in said characters. 

1 16. The article of claim 15 wherein said medium stores instructions to enable a 

2 computer to recognize a user command and obtain a digital representation of a display image 

3 being displayed at the time the user command is received. 
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1 17. The article of claim 1 5 wherein said medium stores instructions to enable a 

2 computer to locate a telephone number and automatically dial said number. 

1 18. The article of claim 15 wherein said medium stores instructions to enable a 

2 computer to locate an Internet address and automatically contact said Internet address. 

1 19. The article of claim 15 wherein said medium stores instructions to enable a 

2 computer to store said contact information. 

1 20. The article of claim 15 wherein said medium stores instructions to enable a 

2 computer to provide the contact information to the user, allow the user to alter the contact 

3 information and revise the information according to information from the user. 

1 21 . The article of claim 15 wherein said medium stores instructions to enable a 

2 computer to automatically add said contact information to a soft ware address director y. 

1 22. The article of claim 15 wherein said medium stores instructions to enable a 

2 computer to obtain a digital representation of a television image. 
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